Image pickup apparatus

ABSTRACT

A short-time exposure image data and a long-time exposure image data of the same object to be photographed are stored to SE memory and LE memory and then transferred to CPU, motion thereof being detected at a motion detecting section on the basis of the two image data; if no motion is detected, a wide dynamic range, synthesized image as synthesized at a synthesizing circuit on the basis of image data read out from the two memories and compressed is outputted through a selector controlled by CPU. If a motion is detected, the long-time exposure image data read out from the LE memory and processed at a signal processing circuit is outputted through the selector. An image pickup apparatus having a function for generating wide dynamic range, synthesized image by synthesizing image signals corresponding to a plurality of frames of different exposure amounts is thereby able to perform ON/OFF switching of the generating function of synthesized image on the basis of previously taken image data, so that a failed synthesized image can be prevented from being outputted.

RELATED APPLICATIONS

This application is a divisional of application U.S. patent applicationSer. No. 09/358,657, filed on Jul. 26, 1999, which claims the benefit ofJapanese Application No. 10-226575 filed in Japan on Jul. 28, 1998, thedisclosures of which are incorporated herein by reference in theirentirety.

BACKGROUND OF THE INVENTION

The present invention relates to image pickup apparatus in which imagesignals corresponding to a plurality of frames at different exposureamounts are outputted from an image pickup device and are synthesized toobtain an image having wide dynamic range, and also relates to apparatusrelated thereto.

Solid-state image pickup devices such as CCD image pickup device aregenerally used in image pickup apparatus, as TV camera, video camera,electronic camera, etc. There is a problem however that the dynamicrange of a solid-state image pickup device is much narrower than that ofa silver salt photographic film.

To eliminate this problem, there have been proposed techniques in whichimage signals corresponding to two frames at different exposure amountsare read out from a single image pickup device and are synthesized toobtain an image having an increased dynamic range. For example, an imagepickup apparatus having the following construction is disclosed inJapanese Patent Publication No. 2522015. In particular, a disclosure hasbeen made with respect to an image pickup apparatus including: imagepickup means for converting object image into electrical signals; imagepickup control means for selectively executing a first mode or secondmode, said first mode for successively outputting images at differentexposure amounts in cycles by cyclically changing accumulating time atthe image pickup means, image portions at suitable level beingsynthesized to form a synthesized frame of said predetermined cycle byrespectively comparing signal level of each portion of a plurality offrames at different exposure amounts outputted from said image pickupmeans with a predetermined reference value, said second mode forproviding output by using a constant accumulating time at said imagepickup means; and switch control means for switching the image pickupcontrol means to the first mode when an existence of object having alarge difference in luminance within a frame is detected in said secondmode by comparing signals obtained from said image pickup means with apredetermined signal level. This image pickup apparatus, when in thefirst mode, is capable of obtaining an image of which all the portionsof the frame are at suitable signal level. Even when an object having alarge difference in luminance exists within a frame as in a backlightedcondition, lack of detail at high level portion for example does notoccur within the frame and it is in effect possible to widen dynamicrange thereof.

Further, without a determination made by the operator on the conditionsof the object, etc., switching to the first mode is automatically madeby the switching control means when an object having a large differencein luminance exists within the frame in the second mode, making itpossible to automatically correct a backlighted condition, etc.

Further, a disclosure has been made in Japanese patent application laidopen No. 5-64075 with respect to a video camera including: image pickupmeans including a solid-state image pickup device, for outputting imagesignals for representing an object image; motion detection means fordetermining based on the image signals obtained from the image pickupmeans whether a motion exceeding a predetermined amount occurs in theobject or not; means for controlling exposure amount so that the imagepickup means takes images of the object at two different exposureamounts on condition that a motion has not been detected by the motiondetection means; and means for forming synthesized image signals byreplacing, of the image signals representing an image at the largerexposure amount, those image signals representing regions of relativelyhigh luminance in that image with those image signals representing thecorresponding regions in an image at the smaller exposure amount. Also,the same publication discloses a video camera which furthermore includeshigh luminance region detection means for determining whether or not, ofthe above described image signals for representing an image at thelarger exposure amount, the regions of relatively high luminance in thatimage exceed a predetermined area. The above described exposure amountcontrol means controls the exposure amount so that images can be pickedup at two different exposure amounts on condition that no motion hasbeen detected by the above motion detection means and that a highluminance region has been detected by the above described high luminanceregion detection means.

In the above described image pickup apparatus as disclosed in JapanesePatent Publication No. 2522015, however, the mode switching means forswitching to the first mode for generating image having a wide dynamicrange is to switch the mode when an existence of object having a largedifference in luminance is detected within a frame obtained in thesecond mode. The disclosure merely refers to the switching of modedepending on the conditions only of luminance values and otherconditions concerning the generation of wide dynamic range image werenot at all taken into consideration.

Further, in Japanese patent application laid open No. 5-64075, motion ofthe object is detected from image signals before taking of large and asynthesized image is obtained by picking up images at two differentexposure amounts on condition that motion of the object is absent. Thusthe detecting of motion of the object from something other than theimage signals before image taking was not taken into consideration.Other conditions for generating a synthesized image except the detectionof motion of the object were not taken into consideration either.

SUMMARY OF THE INVENTION

The present invention has been made to eliminate the above describedproblems in the conventional image pickup apparatus having a functionfor synthesizing two image signals of different exposure amounts togenerate wide dynamic range, synthesized image. Its main object is toprovide an image pickup apparatus in which ON/OFF switching ofprocessing for the generation of wide dynamic range, synthesized imagecan be performed at high accuracy.

In a first aspect of the invention, there is provided an image pickupapparatus including: image pickup means capable of taking images of thesame object at a plurality of different exposure amounts to generateimage signals corresponding to a plurality of frames at differentexposure amounts; and means for generating wide dynamic range,synthesized image by synthesizing image signals corresponding to aplurality of frames at different exposure amounts obtained by the imagepickup means. It further includes means for determining whether or not asuitable synthesized image can be obtained on the basis of image signalscorresponding to a plurality of previously taken frames; and means forcontrolling ON/OFF switching of the generation processing operation atthe means for generating synthesized image based on an output of thedetermination means.

Since ON/OFF switching control of the generation processing operation atthe means for generating wide dynamic range, synthesized image is thusperformed based on image signals corresponding to a plurality ofpreviously taken frames, an image pickup apparatus can be achieved ascapable of accurately performing ON/OFF switching control of processingoperation for the generation of wide dynamic range, synthesized image.The above main object is thereby accomplished.

It is another object of the present invention to provide an image pickupapparatus in which, when the object has been moved, it is possible toprevent a failure of wide dynamic range, synthesized image.

In a second aspect of the invention, the determination means of theimage pickup apparatus according to the first aspect includes means fordetecting motion in the object image and the generation processingoperation at the means for generating synthesized image is switched toOFF when motion in the object image has been detected by the motiondetection means.

Since the motion detection means is thus provided so as not to performprocessing operation for the generation of synthesized image if a motionin the object image has been detected, it is possible to prevent afailure of wide dynamic range, synthesized image when the object hasbeen moved. The above object is thereby accomplished.

It is still another object of the invention to provide an image pickupapparatus in which the feasible range for image taking can be madegreater with respect to the conditions of object of which a wide dynamicrange, synthesized image can be formed.

In a third aspect of the invention, the determination means of the imagepickup apparatus according to the first aspect includes means fordetecting motion in the object image and motion amount determinationmeans for determining whether or not the amount of motion detected bythe motion detection means is within an allowable range for correction.The apparatus further includes motion correction means for, when theamount of motion is determined as within the allowable range forcorrection by the motion amount determination means, correcting theamount of motion based on such determination output so that synthesizingprocess be performed with correcting the amount of motion of the imagesignals corresponding to a plurality of frames.

Since the motion detection means and the motion amount determination andthe motion correction means are thus provided so as to performsynthesizing process with correcting the amount of motion when themotion amount is within the allowable range for correction, it ispossible to provide a grater range for the conditions of object of whicha wide dynamic range, synthesized image can be formed. The above objectis thereby accomplished.

It is yet another object of the invention to provide an image pickupapparatus capable of taking images suitable for the generation of widedynamic range, synthesized image without requiring detection of theconditions for generating wide dynamic range, synthesized image afterthe taking of image.

In a fourth aspect of the invention, there is provided an image pickupapparatus including: image pickup means capable of taking images of thesame object at a plurality of different exposure amounts to generateimage signals corresponding to a plurality of frames at differentexposure amounts; and means for generating wide dynamic range,synthesized image by synthesizing image signals corresponding to aplurality of frames at different exposure amounts obtained by the imagepickup means. It further includes means for determining whether or not asuitable synthesized image can be obtained on the basis of informationof the object obtained before the taking of image; and means forcontrolling ON/OFF switching of the generation processing operation atthe means for generating synthesized image based on an output of thedetermination means.

ON/OFF switching of processing operation for the generation of widedynamic range, synthesized image can thus be effected relying on objectinformation obtained before the taking of image such as the informationon detection of motion of the object which is obtained based on anoutput from an external AF detection circuit or external AE detectioncircuit. An image pickup apparatus can be achieved as capable of takingimages suitable for the generation of wide dynamic range, synthesizedimage. The above object is thereby accomplished.

It is a further object of the invention to provide an image pickupapparatus adapted to be capable of readily generating wide dynamicrange, synthesized image even of an object involving a motion owing tothe fact that the motion of the object can in effect be stopped bystrobe image taking.

In a fifth aspect of the invention, an image pickup apparatus accordingto the fourth aspect includes strobe emission means. In addition, thedetermination means thereof includes means for detecting motion in theobject image so that, when a motion in the object image has beendetected by the motion detection means, images are taken in a forcedstrobe emission mode to perform the generation processing of synthesizedimage.

Since images are thus taken in a forced strobe emission mode and thegeneration processing of synthesized image is performed when a motion inthe object has been detected at the motion detection means, the motionin the object can in effect be stopped even of an object involving amotion, making it possible to readily generate a wide dynamic range,synthesized image. The above object is thereby accomplished.

It is a further object of the invention to provide an image pickupapparatus adapted to be capable of also readily generating wide dynamicrange, synthesized image even of an object involving a motion owing tothe fact that the motion in the object can in effect be reduced by usinga higher shutter speed.

In a sixth aspect of the invention, an image pickup apparatus accordingto the fourth aspect includes a shutter speed/diaphragm control means.In addition, the determination means thereof includes means fordetecting motion in the object image so that, when a motion in theobject image has been detected by the motion detection means, images aretaken by using a higher shutter speed to perform the generationprocessing of synthesized image.

Since the generation processing of synthesized image is thus performedat a higher shutter speed when a motion in the object image has beendetected, the motion of the object in image signals corresponding to aplurality of frames can be reduced. Since the motion of the object canin effect be reduced even of an object involving a motion by using thehigher shutter speed, it becomes possible to readily generate a widedynamic range, synthesized image. The above object is therebyaccomplished.

It is a further object of the invention to provide an image pickupapparatus capable of generating a suitable wide dynamic range,synthesized image by detecting a backlighted condition.

In a seventh aspect of the invention, the determination means of theimage pickup apparatus according to the fourth aspect includes an AEdetection circuit and the generation processing of synthesized image isperformed when a backlighted condition has been detected by the AEdetection circuit.

Upon the detection of a backlighted condition at the AE detectioncircuit, since the condition for wide dynamic range prevails at the timeof the backlighted condition, a suitable synthesized image can beobtained by performing the generation processing of wide dynamic range,synthesized image. The above object is thereby accomplished.

It is a further object of the present invention to provide an imagepickup apparatus in which images can be readily taken in a mannersuitable for the generation of wide dynamic range, synthesized imagewithout requiring detection of the conditions for the generation of widedynamic range, synthesized image from taken images.

In an eighth aspect of the invention, there is provided an image pickupapparatus including: image pickup means capable of taking images of thesame object at a plurality of different exposure amounts to generateimage signals corresponding to a plurality of frames at differentexposure amounts; and means for generating wide dynamic range,synthesized image by synthesizing image signals corresponding to aplurality of frames at different exposure amounts obtained by the imagepickup means. It further includes means for determining whether or not asuitable synthesized image can be obtained on the basis of informationset in the image pickup apparatus before the taking of image; and meansfor controlling ON/OFF switching of the generation processing operationat the means for generating synthesized image based on an output of thedetermination means.

Since ON/OFF switching of the generation processing operation at themeans for generating wide dynamic range, synthesized image is thuscontrolled based on information set in the image pickup apparatus beforethe taking of image, an image pickup apparatus can be achieved ascapable of readily taking images in a manner suitable for the generationof wide dynamic range, synthesized image without requiring detection ofthe conditions for the generation of wide dynamic range, synthesizedimage from taken images. The above object is thereby accomplished.

It is a further object of the present invention to provide an imagepickup apparatus in which it is possible to make a determination basedon the shutter speed and lens zoom information as to whether or not asuitable synthesized image can be generated.

In a ninth aspect of the invention, information set in the image pickupapparatus according to the eighth aspect is shutter speed informationand/or lens zoom information and the determination means makes adetermination based on the shutter speed and/or lens zoom information.

By constructing in this manner, ON/OFF switching of the generationprocessing operation of synthesized image can be performed in linkagewith the shutter speed and lens zoom information. When the shutter speedis slow or when the zoom ratio is high, the generation processingoperation of synthesized image can be turned OFF to avoid a failure inthe synthesized image which occurs due to a movement of the object. Theabove object is thereby accomplished.

It is a further object of the present invention to provide an imagepickup apparatus having a function for generating wide dynamic range,synthesized image by synthesizing a plurality of image signals ofdifferent exposure amounts, in which a suitable synthesized image isalways obtained even when a shift from set exposure amounts hasoccurred.

In a tenth aspect of the invention, there is provided an image pickupapparatus including: image pickup means capable of taking images of thesame object at a plurality of different exposure amounts to generateimage signals corresponding to a plurality of frames at differentexposure amounts; and means for generating wide dynamic range,synthesized image by synthesizing image signals corresponding to aplurality of frames at different exposure amounts obtained by the imagepickup means. It further includes: means for correcting exposure amountsof the image signals corresponding to a plurality of frames of differentexposure amounts; and switching control means for performing ON/OFFswitching of the correction operation at the exposure amount correctionmeans based on an image taking mode set at the image pickup apparatus.

By thus providing the means for correcting exposure amounts of imagesignals and ON/OFF switching control means based on the image takingmode at the exposure amount correction means, the exposure amounts canbe corrected to obtain a suitable synthesized image only when there is apossibility of shift from the set exposure amounts of the image signalscorresponding to a plurality of frames of different exposure amounts.The above object is thereby accomplished.

It is a further object of the invention to provide an image pickupapparatus in which a suitable synthesized image can be obtained evenwhen using a strobe.

In an eleventh aspect of the invention, the image pickup apparatusaccording to the tenth aspect further includes strobe emission means.The switching control means controls switching so that the exposureamount correction means is caused to operate when the apparatus has beenset to a strobe image taking mode.

While a variance tends to occur in the emission amount of a strobe, asuitable synthesized image for example without a discontinuity ingradation can be generated even when images are taken by using strobe,since the exposure amount correction means is forced to operate at thetime of the strobe image taking mode. The above object is therebyaccomplished.

It is a further object of the invention to provide an image synthesizingsystem capable of generating wide dynamic range, synthesized image inwhich image taking interval can be reduced without requiring acomplicated processing at the image pickup apparatus thereof.

In a twelfth aspect of the invention, there is provided an imagesynthesizing system, including: an image pickup apparatus having imagepickup means capable of taking images of the same object at a pluralityof different exposure amounts to generate image signals corresponding toa plurality of frames at different exposure amounts and recording meansfor recording as a single file to a recording medium the image signalscorresponding to a plurality of frames obtained by the image pickupmeans and synthesizing information to be used in generating a widedynamic range, synthesized image by synthesizing the image signalscorresponding to the plurality of frames; and an external synthesizingapparatus having synthesizing means for synthesizing the image signalscorresponding to a plurality of frames within a file recorded in therecording medium based on the synthesizing information in the file.

By thus performing synthesizing process at an external synthesizingapparatus externally connected to the image pickup apparatus, it becomesunnecessary to perform a complicated synthesizing process within theimage pickup apparatus. Time for processing of image taking is shortenedand it becomes possible to reduce the image taking interval. The aboveobject is thereby accomplished.

It is a further object of the present invention to provide an imagesynthesizing system in which image signals corresponding to one framecan be recorded when the remaining capacity of the recording medium isless than that for recording image signals corresponding to a pluralityof frames.

In a thirteenth aspect of the invention, the image synthesizing systemaccording to the twelfth aspect further includes detection means fordetecting the remaining capacity of the recording medium. The remainingcapacity of the recording medium is detected by the detection means and,if not enough remaining capacity left in the recording medium forrecording image signals corresponding to a plurality of frames ofdifferent exposure amounts, image signals corresponding to one frame isobtained by taking image of the object at a predetermined exposureamount by the image pickup means and such image signals corresponding toone frame is recorded to the recording medium by the recording means.

By such construction, an ordinary image taking corresponding to oneframe can be performed even when the remaining capacity of the recordingmedium is not enough for recording image signals corresponding to aplurality of frames. The above object is thereby accomplished.

It is a further object of the present invention to provide an imagepickup apparatus in which, when image signals corresponding to aplurality of frames for generating a wide dynamic range, synthesizedimage are to be recorded in a recording medium, synthesizing processthereof can be performed by treating them in a similar manner as imagesignals corresponding to an ordinary one frame.

In a fourteenth aspect of the invention, there is provided an imagepickup apparatus which includes: image pickup means capable of takingimages of the same object at a plurality of different exposure amountsto generate image signals corresponding to a plurality of frames atdifferent exposure amounts; and recording means for recording as asingle file to a recording medium the image signals corresponding to aplurality of frames obtained by the image pickup means and synthesizinginformation to be used in generating wide dynamic range, synthesizedimage by synthesizing the image signals corresponding to the pluralityof frames.

Since image signals corresponding to a plurality of frames and thesynthesizing information thereof are recorded as a single file to therecording medium, it is possible to perform synthesizing process bytreating them similarly as in the case of image signals corresponding toan ordinary single frame. The above object is thereby accomplished.

It is a further object of the present invention to provide an externalsynthesizing apparatus in which, when a wide dynamic range, synthesizedimage is generated by synthesizing image signals corresponding to aplurality of frames, the synthesized image can be generated similarly asan ordinary single image signal without a special operation.

In a fifteenth aspect of the invention, there is provided an externalsynthesizing apparatus including synthesizing means for reading a fileout from a recording medium, said file being recorded in said recordingmedium as a single file recording image signals corresponding to aplurality of frames of different exposure amounts obtained by takingimages of the same object at a plurality of different exposure amountsand synthesizing information to be used in generating wide dynamicrange, synthesized image by synthesizing the image signals correspondingto the plurality of frames, said synthesizing means generating widedynamic range, synthesized image by synthesizing the image signalscorresponding to the plurality of frames contained in the file based onthe synthesizing information contained in the file.

Since image signals corresponding to a plurality of frames andsynthesizing information thereof are thus recorded as a single file in arecording medium from which said file is read out to effect synthesizingprocess, a wide dynamic range synthesized image can be outputtedsimilarly as in the case of an ordinary taken image without a specialoperation. The above object is thereby accomplished.

It is a further object of the invention to provide a recording mediumrecorded in which includes a program capable of readily performingsynthesis of image signals corresponding to a plurality of framesobtained by taking images of the same object at a plurality of differentexposure amounts.

A recording medium recording a program for performing synthesis of imagesignals corresponding to a plurality of frames obtained by taking imagesof the same object at a plurality of different exposure amounts isprovided in accordance with a sixteenth aspect of the invention, therecording medium recording, as a single file, the image signalscorresponding to the plurality of frames of different exposure amountsand synthesizing information to be used in generating a wide dynamicrange, synthesized image by synthesizing the image signals correspondingto the plurality of frames and at the same time recording a program forachieving a function for reading out the file to synthesize the imagesignals corresponding to the plurality of frames contained in the filebased on the synthesizing information contained in the file.

By constructing in this manner, a recording medium can be provided asthat recording a program by which synthesis can be readily performed ofimage signals corresponding to a plurality of frames obtained by takingimages of the same object at a plurality of different exposure amounts.The above object is thereby accomplished.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing an overall construction of a firstembodiment of the image pickup apparatus according to the presentinvention.

FIG. 2 is a block diagram showing an example of construction of thecamera signal processing circuit in the embodiment shown in FIG. 1.

FIGS. 3A to 3D illustrate the manner of detecting motion at the motiondetecting section in the first embodiment.

FIG.4 explains the manner of correcting a motion by the motioncorrecting circuit.

FIGS. 5A, 5B, 5C are flowcharts explaining camera control operation forperforming ON/OFF switching of the generation processing of wide dynamicrange, synthesized image based on previously taken image data.

FIG. 6 is a block diagram showing main portions of a second embodimentof the present invention.

FIG. 7 is a block diagram showing an example of construction of camerasignal processing circuit of the second embodiment shown in FIG. 6.

FIG. 8 shows the manner of AF processing at an external AF circuit.

FIG. 9 is a block diagram showing main portions of a modification of thesecond embodiment shown in FIG. 6.

FIG. 10 shows the manner of AE processing at an external AE circuit.

FIG. 11 is a block diagram showing main portions of another modificationof the second embodiment.

FIG. 12 is a block diagram showing main portions of a third embodimentof the present invention.

FIG. 13 shows a program diagram of shutter speed/aperture stop.

FIG. 14 explains the manner of correcting exposure amount.

FIG. 15 shows the file structuring of image data and synthesizinginformation to be used in their synthesizing process.

FIG. 16 is a flowchart explaining the processing operation in causing anexternally connected personal computer to perform synthesizing process.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

Some embodiments of the present invention will now be described. First,a description will be given below by way of a block diagram shown inFIG. 1 with respect to an overall construction of a first embodiment ofthe electronic camera to which the present invention is applied.Referring to FIG. 1, denoted by numeral 1 is a single-plate (sensor)color CCD image pickup device for photoelectrically converting anoptical signal into an electrical signal. It includes an electronicshutter function. An object light is inputted to CCD image pickup device1 through a lens 2 and a stop/shutter mechanism 3. An output of CCDimage pickup device 1 is amplified at an amplifier 4 after removed ofnoise for example at a correlation double sampling circuit. Ananalog-to-digital converter 5 converts the output of the amplifier 4outputted as analog data into digital data. A camera signal processingcircuit 6 processes signals from CCD image pickup device 1 as a videodata. Those denoted by numeral 7 include AF detection circuit forextracting AF (autofocus) information to control focus, an AE detectioncircuit for extracting AE (auto exposure) information to controlexposure, and AWB detection circuit for extracting AWB (auto whitebalance) information to set white balance, by using image pickup signalsetc., from the CCD image pickup device 1 prior to the main taking ofimage. Output signals from the AF, AE, AWB detection circuit 7 suppliesthrough CPU 8 AF information to lens 2, AE information to stop/shuttermechanism 3, and AWB information to the camera signal processing circuit6.

Numeral 9 denotes a compression circuit (JPEG) for compressing theamount of data. The image data compressed at the compression circuit 9is recorded at memory card 15 through memory card I/F 14. Memorycontroller 10 and DRAM 11 are used as working memory when performingcolor processing, etc., of video data. A display circuit 12 and LCDdisplay unit 13 are used for example to confirm the image takingconditions by reading out and displaying data recorded at the memorycard 15. Denoted by numeral 16 is a personal computer I/F fortransferring data recorded on the memory card 15 to a personal computer17. Further, denoted by numeral 19 is a printer for providing a printoutput of image pickup data recorded on the memory card 15 and 18 is aprinter I/F to be used in transferring data to the printer 19. It shouldbe noted that, in FIG. 1, numeral 20 denotes a timing generator whichgenerates timing pulse for driving the CCD image pickup device 1. Itdrives the CCD image pickup device 1 according to control of CPU 8.Numeral 21 denotes a strobe mechanism which is controlled through CPU 8by AE information to be obtained before the main taking of image andeffects control as to whether strobe is to be emitted or not and controlof light amount of the strobe emission. Numeral 22 denotes input keys ofCPU by which setting of various types of image taking mode, driving oftrigger switch, etc., can be performed.

Operation of an electronic camera having the above construction issummarized as follows. Image pickup signals generated by means of twotimes of image taking under different exposure conditions by using theCCD image pickup device 1 are respectively converted into digitalsignals at the analog-to-digital converter 5 and are subjected to apredetermined synthesizing operation processing at the camera signalprocessing circuit 6 to obtain a single synthesized image data which hasa wide dynamic range as a whole. The synthesized image data iscompressed at the compression circuit 9 and is recorded to the memorycard 15.

A construction of the camera signal processing circuit 6 in anembodiment shown in FIG. 1 will now be described by way of the blockdiagram of FIG. 2. FIG. 2 includes: 6-1, SE memory for storingshort-time exposure image data; 6-2, LE memory for storing long-timeexposure image data; 6-3, a multiplier for multiplying a short-timeexposure image data read out from the SE memory 6-1 by an exposureamount ratio A (=LE/SE) of the short-time exposure image and long-timeexposure image; 6-4, a synthesizing circuit for forming by means ofsynthesis a wide dynamic range, synthesized image from the short-timeexposure image data after the multiplication and the long-time exposureimage data read out from the LE memory 6-2; 6-5, a compression circuitfor compressing the synthesized image obtained at the synthesizingcircuit 6-4; 6-6, a signal processing circuit for subjecting thelong-time exposure image data read out from the LE memory 6-2 to suchprocessing as γ-correction and edge enhancement; and 6-7, a selector forproviding an output by switching based on control signal from CPU 8between the synthesized image from the synthesizing circuit 6-4 and thelong-time exposure image data having been subjected to signal processingat the signal processing circuit 6-6. Further, denoted by numeral 8A isa motion detecting section provided internally of CPU 8, for detectingmotion in the object based on the short-time exposure image data andlong-time exposure image data read out from the SE memory 6-1 and LEmemory 6-2. Switching of the selector 6-7 is controlled on the basis ofthe output of the motion detecting section 8A.

A description will now be given with respect to an operation of thusconstructed camera signal processing circuit 6. First, short-timeexposure image data and a long-time exposure image data of the sameobject taken at the CCD image pickup device 1 are temporarily stored toSE memory 6-1 and LE memory 6-2. The image data are then transmittedinto CPU 8 from the two memories 6-1, 6-2. A motion of the object isdetected at the motion detecting section 8A based on the two image dataof different exposure amounts. If no motion is detected at the motiondetecting section 8A, a wide dynamic range, synthesized imagesynthesized at the synthesizing circuit 6-4 based on the image data readout from the two memories 6-1, 6-2 and compressed at the compressioncircuit 6-5 is outputted through the selector 6-7 which is controlled byCPU 8. On the other hand, if a motion is detected, the long-timeexposure image data read out from the LE memory 6-2 and subjected tosignal processing at the signal processing circuit 6-6 is outputtedthrough the selector 6-7. A motion is thus detected on the basis ofpreviously taken image data so that it is possible to prevent a failed,synthesized image from being outputted when the object is moved at thetime of taking two images at different exposure amounts.

Actual motion detecting techniques to be performed at the motiondetecting section 8A provided within CPU 8 will now be described by wayof FIGS. 3A to 3D. There are two methods for detecting motion frompreviously taken two frames of image data. In a first method, adifference is obtained of a short-time exposure image data as shown inFIG. 3A (object 31 a having a lack of detail at low level portion andbackground 31 b taken at a suitable exposure in this short-time exposureimage data) from a long-time exposure image data as shown in FIG. 3B(object 32 a taken at a suitable exposure but background 32 b lackingdetail at its high level portion in this long-time exposure image data).If no motion occurs in the object, such difference becomes zero and anabsence of motion is detected. On the other hand, if there is a motion,difference at the portion of occurrence of motion does not become zeroas shown in FIG. 3C and a motion is thereby detected. It should be notedthat, when obtaining difference between the two image data of differentexposure amounts, the short-time exposure image data SE is multiplied bythe exposure amount ratio A and its difference (LE−SE×A) from thelong-time exposure image data LE is obtained. Referring to FIG. 3C, theslants represent the portion where the difference is “+” while thedotted region represents the portion where the difference is “−”.

In another method for detecting motion, it is detected from ratio of thetwo image data of different exposure amounts. The exposure amount ratioof the two image data, A=LE/SE, is represented by α·A. When a motion asshown in FIG. 3D occurs, the slant portion represents a region where α>1and the dotted portion represents a region where α<1, both indicatingthe moving portion. It is determined as an occurrence of motion of theobject when these regions are detected.

A modification of the first embodiment will now be described. In thismodification, when the amount of motion detected at the motion detectingsection 8A is relatively small and is determined as within a range ofallowable amount for correction by CPU 8, the motion is corrected andsynthesizing process for wide dynamic range, synthesized image isperformed. The motion correction circuit in this case is constructed asincluded in the synthesizing circuit 6-4.

In the method of correction by the motion correction circuit, if aregion indicated by X is the region where motion is to be corrected whenregions along a horizontal direction are taken into consideration asshown in FIG. 4, values are used of pixels A, B in regions without anoccurrence of motion, at a suitable distance d for performing correctionfrom the two ends of the region X to be corrected of motion. The A, Bpixels are determined as at two ends and the motion is corrected byinterpolating the motion correction region X so as to uniformly changepixel values between the two.

It should be noted that, since the region (object portion) withoutlacking detail in its highlighted portion in the long-time exposureimage is used as it is for synthesis in generating a synthesized image,the “−” region where motion occurs as shown in FIG. 3C is not correctedand only the “+” region [i.e., region of LE−(SE×A)>0] is subjected tothe motion correction. Further, as shown in FIG. 3D, since the region ofα<1 is used as it is in the synthesizing process, a motion correctionthereof is not performed and only the region of α>1 is corrected ofmotion.

Shown in FIGS. 5A to 5C are flowcharts collectively representing acamera control operation in the above embodiment for effecting ON/OFFswitching of the output of wide dynamic range, synthesized image on thebasis of motion in previously taken image data, etc. FIG. 5A is theflowchart of a broader term of this embodiment showing camera controloperation where ON/OFF switching of the synthesized image output isperformed by determining based on previously taken image data whether ornot the wide dynamic range, synthesized image is suitable. At first, inthis case, AE, AF control is performed based on AE, AF informationobtained before the main taking of image (step S1). An image is thentaken twice under different exposure conditions (noted as SL imagetaking) to generate two image data (step S2). It is then determinedwhether wide dynamic range, synthesized image (noted as SL synthesizedimage) will be suitable or not on the basis of the two image data (stepS3). If the wide dynamic range, synthesized image is suitable, the widedynamic range, synthesized image is outputted (step S4). If thesynthesized image is unsuitable, the image data of long-time exposure(noted as LE image) is outputted (step S5).

FIG. 5B is the flowchart showing a camera control operation in the caseof detecting motion of the object from two image data under differentexposure conditions to determine whether wide dynamic range, synthesizedimage will be suitable or not. In particular, after generating two imagedata of different exposure amounts at the same step, motion of theobject is detected on the basis of the two image data (step S11). Ifmotion of the object is not detected, a wide dynamic range, synthesizedimage is outputted (step S12). If motion of the object is detected, thewide dynamic range, synthesized image is determined as unsuitable andthe long-time exposure image data is outputted (step S13).

FIG. 5C is a flowchart showing a camera control operation where, evenwhen motion has been detected by detecting motion of the object from twoimage data under different exposure conditions, motion is corrected ifthe amount of motion is within an allowable range for correction. Inparticular, motion of the object is similarly detected on the basis ofthe two image data of different exposure amounts and, if motion isdetected, it is determined whether the amount of the motion is within anallowable range for correction or not (step S14). If, then, the amountof motion is smaller than an allowable amount for correction, a widedynamic range, synthesized image after corrected of motion is outputted(step S15). If the amount of motion is greater than the allowable amountfor correction, the image data of the long-time exposure is outputted(step S16).

A second embodiment will now be described. In this embodiment, cameracontrol including ON/OFF switching of the generation and outputting ofwide dynamic range, synthesized image is performed on the basis ofinformation obtained before the taking of image. FIG. 6 is a blockdiagram showing main portions of the second embodiment, where likecomponents as in the first embodiment shown in FIG. 1 are denoted bylike reference numerals. This is different from the first embodimentshown in FIG. 1 in that an external AF circuit 23 is provided and CPU 8includes a motion detecting section 8B for detecting motion of theobject on the basis of AF signals from the external AF circuit 23.Further, as shown in FIG. 7, the only frame memory included in thecamera signal processing circuit 6′ is SE memory 6-1 for storingshort-time exposure image data. The short-time exposure image data istaken into SE memory 6-1. The short-time exposure image data is read outfrom SE memory 6-1 in accordance with timing of the long-time exposureimage data and, after multiplication processing performed based on theexposure amount ratio, synthesizing process is performed at thesynthesizing circuit 6-4 and compression processing is performed at thecompression circuit 6-5. Furthermore, the AE, AF, AWB detection circuit7 in the first embodiment is replaced by an AE, AWB detection circuit7′.

Those which may be used as the external AF circuit 23 for exampleincludes an AF sensor capable of finding range of a plurality of points(five points in the illustrated example) on the object as shown in FIG.8. AF operation is performed twice and range-finding information isinputted to the motion detecting section 8B which is provided internallyof CPU 8. It is determined at the motion detection circuit 8B as thatthe object is moving when a change occurs in distance informationobtained by the two times of AF operation for any one of the pluralityof range-finding points. The image is taken only at a long-time exposurebased on the second distance information and, without performing thegeneration processing of synthesized image, only the image data of thelong-time exposure is outputted.

In other words, in the second embodiment, motion of the object isdetected at the motion detecting section 8B internally provided of CPU,based on AF signals from the external AF circuit. If no motion isdetected, the selector 6-7 is previously switched at the camera signalprocessing circuit 6′ so as to output a synthesized and compressed widedynamic range image. If motion is detected, on the other hand, theselector 6-7 is previously switched so that a long-time exposure imagedata taken by the second AF information is outputted to prevent asynthesized image failed due to motion from being outputted.

Since ON/OFF switching of the outputting of synthesized image is thuspreviously performed based on AF information obtained before the takingof image from an external AF circuit, it is not necessary to drive CCDimage pickup device in order to perform ON/OFF switching control of thesynthesized image output, thereby reducing dissipation power. Further,it is not necessary to detect a motion on the basis of image data afterthe taking of image so that images can be taken in a manner suitable forthe generation of wide dynamic range, synthesized image from thebeginning.

A modification of the present embodiment will now be described. In theabove second embodiment, the external AF circuit is provided and motionof the object is detected on the basis of AF signals. In thismodification, however, an external AE circuit 24 is provided as shown inFIG. 9 instead of the external AF circuit. A motion of the object isdetected by a motion detecting section 8C provided in CPU 8 on the basisof two times of AE signals from the external AE circuit 24. In a similarmanner as in the second embodiment based on such motion detectionsignals, if no motion is detected, the selector 6-7 of the camera signalprocessing circuit 6′ shown in FIG. 7 is previously switched so as tooutput a wide dynamic range, synthesized image which has beensynthesized and compressed. If a motion is detected, the selector 6-7 atthe camera signal processing circuit 6′ is also previously switched sothat image data of long-time exposure based on the second AE informationis outputted. It should be noted that denoted by 7″ in FIG. 9 is an AF,AWB detection circuit.

Those which may be used as the external AE circuit 24 for exampleincludes an AE sensor capable of photometry by dividing the object intoa plurality of portions as shown in FIG. 10. AE operation is performedtwice and exposure information is inputted to the motion detectingsection 8C internally located at CPU 8. It is determined at the motiondetecting section 8C as that the object is moving and image is taken oflong-time exposure by the second exposure information when a change inexposure information occurs in any one of the plurality of divisionalphotometric regions between the two times of AE operation.

As the above, in this modification, too, motion of the object isdetected at the motion detecting section 8C internally provided of CPUon the basis of AE information from the external AE circuit. When nomotion is detected, the selector 6-7 of the camera signal processingcircuit 6′ is previously switched so that a wide dynamic range,synthesized image after synthesis and compression is outputted. If amotion is detected, on the other hand, the selector 6-7 is previouslyswitched so that image data of long-time exposure taken by the second AEinformation is outputted to prevent a synthesized image failed due tomotion from being outputted. Accordingly, since ON/OFF switching of thesynthesized image output can be performed without driving the imagepickup device, it is possible to reduce dissipation power and it is notnecessary to detect motion on the basis of image data after the takingof image so that images can be taken in a manner suitable for thegeneration processing of synthesized image from the beginning.

A further modification will now be described. In this modification, asshown in FIG. 11, a camera shake detecting circuit 25 for example usingan acceleration sensor is provided so that, when camera shake isdetected, an image taking for the generation of synthesized image is notperformed and image is taken anew. In other words, an occurrence ofcamera shake causes a difference between the two image data of differentexposure amounts; if these are synthesized, an unsuitable synthesizedimage results. Accordingly, image is taken anew in case of camera shakeso that it is possible to prevent a failed synthesized image from beingoutputted.

A third embodiment will now be described. FIG. 12 is a block diagramshowing main portions of the third embodiment. In this embodiment, amotion detection circuit 26 is provided so that motion of the object ispreviously detected on the basis of image pickup data which is obtainedfrom CCD image pickup device 1 before the main taking of image. When amotion has been detected, a wide dynamic range, synthesized image isgenerated by taking image twice at different exposure amounts (emissionamount) by using a forced strobe emission.

The motion detection circuit 26 includes a memory for image datacorresponding to two frames so that two image data of the same objectpreviously obtained from CCD image pickup device 1 are alternatelystored to the memory to detect motion of the object by using such knowntechniques as frame correlation. If a motion has been detected at themotion detection circuit 26, image is taken twice in a short time periodwith causing the strobe 21 through CPU 8 to emit twice by changingemission amount in such a manner as to correspond to an exposure amountratio for obtaining a wide dynamic range image. A wide dynamic range,synthesized image is then generated by performing synthesizing process.Further, if motion is not detected, image is taken twice at differentexposure amounts without using a forced emission of the strobe 21 toperform the generation processing of wide dynamic range, synthesizedimage. Accordingly, in this embodiment, the camera signal processingcircuit does not require a selector as that shown in FIG. 2.

The generation processing of wide dynamic range, synthesized images isperformed by thus using a forced strobe mode correspondingly to themotion of the object. Since, thereby, images can be taken with stoppingthe motion of the object for the two times of strobe image taking in ashort time period, it is possible to prevent a failed synthesized imagefrom being outputted.

This embodiment has been shown as that in which motion of the object isdetected by using image pickup data from the CCD image pickup device. Asshown in FIG. 6 or 9, however, it is also possible to detect motion at amotion detecting section provided internally of CPU from AF, AEinformation obtained before the taking of image by using an external AFcircuit or external AE circuit, thereby causing the generationprocessing of synthesized image to be performed in a forced strobe mode.

A modification of the third embodiment will now be described. In thismodification, when a motion of the object has been detected at themotion detecting circuit 26, a higher shutter speed is used instead ofusing a strobe emission. Exposure time is thereby reduced for both theshort-time exposure image taking and the long-time exposure image takingso that the generation processing of wide dynamic range, synthesizedimage is performed by taking two images at different exposure amounts ina short time period.

In the case where images are taken by using a higher shutter speed asthe above, a program shift for widening the aperture stop correspondingto the shutter speed is performed in order to achieve the same exposureamount even with the reduced exposure time due to the higher shutterspeed. Specifically, in the program diagram of shutter speed/aperturestop as shown in FIG. 13, if a motion has been detected at point “A”where shutter speed is 1/30 and aperture value is F8, a program shift ismade for example by shifting to point “B” where shutter speed is 1/250and aperture is widened to F2.8. In this manner, the generationprocessing of wide dynamic range, synthesized image is performed bytaking image twice at different exposure amounts by using a highershutter speed. It is thereby possible to make smaller the motion of theobject so that a synthesized image with lessened failure can beoutputted.

A fourth embodiment will now be described. In this embodiment, when theobject has been detected as in a backlighted condition by means of AEinformation previously obtained on the basis of image data from CCDimage pickup device before the taking of image or AE informationobtained from an external AE circuit, the generation processing of widedynamic range, synthesized image is performed by taking image twice atdifferent exposure amounts, since image data in a backlighted conditiontends to have a wider dynamic range. In this case, too, since ON/OFFswitching of the generation processing of wide dynamic range,synthesized image is performed on the basis of AE information previouslyobtained before the taking of image, the images can be taken in a mannersuitable for the generation of wide dynamic range, synthesized imagefrom the beginning.

A fifth embodiment will now be described. In this embodiment, cameracontrol including ON/OFF switching of the generation processing of widedynamic range, synthesized image is performed on the basis of suchcamera setting information as the image taking conditions set to theimage pickup apparatus (camera) before the taking of image. Although thefundamental construction of this embodiment is similar to that of thefirst embodiment shown in FIG. 1, its camera signal processing circuitis of similar construction as that used in the second embodiment shownin FIG. 7. Here, camera control including ON/OFF switching of thegeneration processing of wide dynamic range, synthesized image isperformed on the basis of various camera setting information to be setfrom the input key 22 of CPU 8. By thus causing camera control includingthe ON/OFF switching of wide dynamic range, synthesized image to beperformed on the basis of information set to the camera before thetaking of image, it becomes unnecessary to detect motion, etc., based onimage pickup data after the taking of image and images can be taken in amanner suitable for the generation of wide dynamic range, synthesizedimage from the beginning, since ON/OFF switching of the generationprocessing of synthesized image is previously performed.

In a first manner of this embodiment, images are taken for thegeneration of wide dynamic range, synthesized image and the generationprocessing of synthesized image is performed when taking image in astrobe mode where strobe emission is used in combination, therebylinking ON/OFF switching of the synthesizing process to the setting of astrobe mode. In particular, even in the case of an object with motion orthe like, it is possible when taking image in a strobe mode that themotion of the object be stopped and a wide dynamic range, synthesizedimage without a failure be obtained by causing strobe emission so as totake image twice at different exposure amounts in a short time period.

In a second manner of this embodiment, ON/OFF switching of thegeneration processing of wide dynamic range, synthesized image is linkedto the shutter speed to be set. In particular, since a slower shutterspeed results in a greater motion of the object, the generationprocessing of wide dynamic range, synthesized image is turned OFF. Theshutter speed serving as a basis for ON/OFF switching of synthesizingprocess somewhat varies in a camera having zoom depending on zoom ratio.In particular, since a higher zoom ratio magnifies the motion, the setshutter speed for turning OFF the synthesized image generationprocessing is shifted toward a higher shutter speed.

In a third manner of this embodiment, ON/OFF switching of the generationprocessing of wide dynamic range, synthesized image is linked to thesetting of an exposure. If, for example, the user effects spotphotometry in setting exposure, it is most likely the case where anattempt is made to take image with performing spot photometry because ofa backlighted condition. Therefore, if a spot photometry mode is set,images are taken for the generation of a wide dynamic range, synthesizedimage.

Further, when an exposure correction is effected manually in a camerahaving a manual exposure correcting function, since a manual exposurecorrection is most likely attempted in the case where the scene to bephotographed is in a backlighted condition or the dynamic range of theobject is large, images are taken for the generation of wide dynamicrange, synthesized image at the time of manually corrected exposure.

In a fourth manner of this embodiment, ON/OFF switching of thegeneration processing of wide dynamic range, synthesized image is linkedto the image taking mode. With a camera having a consecutivephotographing function, for example, since the taking of image for thegeneration of wide dynamic range, synthesized image cannot be effectedin a consecutive taking mode, the generation processing of synthesizedimage is switched to OFF when the consecutive taking mode has been set.Further, since too much motion is not likely to be involved in a scenery(distant view) photographing mode, the generation processing ofsynthesized image is turned ON when such mode has been set. Furthermore,since a large motion of the object is likely in a sports photographingmode, the generation processing of synthesized image is turned OFF whensuch mode has been set. Moreover, when a macro strobe photographing modehas been set, the generation processing of synthesized image is turnedON similarly to the strobe mode.

Since, in this manner, the generation processing of synthesized image isthus performed only when an image taking mode suitable for thegeneration of wide dynamic range, synthesized image has been set, it ispossible to obtain a wide dynamic range, synthesized image withoutfailure.

A sixth embodiment will now be described. In general, image is takentwice by previously setting exposure amount ratio thereof in a camerahaving a function for generating wide dynamic range, synthesized image.The exposure amount ratio of the two images actually taken, however, isnot necessarily the same as the set value. If they are synthesized insuch case, there is a disadvantage that a discontinuity in gradationtends to occur at the seam between the two images. This embodimentintends to mitigate this problem.

Specifically, in this embodiment, a camera having a function forgenerating wide dynamic range, synthesized image includes exposureamount correction means and ON/OFF switching of the exposure amountcorrection corresponding to shift in the exposure amount ratio iscontrolled on the basis of setting of an image taking mode which maycause a shift in the exposure amount ratio. By this construction, sinceexposure amount correction is performed only when an image taking moderequiring exposure amount correction has been set, it is not necessaryto always detect exposure amount to determine whether an exposure amountcorrection is required or not.

In particular, an exposure amount correction is performed when a strobeimage taking mode has been set. In other words, the exposure amount intaking image by using strobe depends not on the shutter speed but on theemission amount of the strobe. Since the strobe emission amount cannotso accurately be controlled, a considerable variance may occur in theexposure amount. Accordingly, correction of exposure amount is performedwhen a strobe emission mode has been set.

The correcting operation of the exposure amount is performed as follows.Image data with a smaller exposure amount and image data with a greaterexposure amount obtained by two times of image taking using strobeemission are stored to the two memories, i.e., SE memory and LE memoryof the camera signal processing circuit shown in FIG. 2. These aresupplied to CPU and regions are extracted of those without lack ofdetail at low level portion in the SE image data and those without lackof detail at high level portion in the LE image data. Sums ΣSE, ΣLE ofpixel data of the respective extracted regions are obtained. and anactual exposure amount ratio A′ is obtained by the ratio ΣLE/ΣSE of thetwo. As shown in FIG. 14, the synthesizing process is performed bymultiplying the low exposure image data SE and the actual exposureamount ratio A′. A discontinuity in gradation at the seam between thetwo image data is thereby prevented. It should be noted that, in FIG.14, the dotted line represents the case of taking images at a previouslyset exposure amount ratio A, indicating a discontinuity in gradation dueto variance in the exposure amount ratio.

A seventh embodiment will now be described. The above embodiments havebeen shown as those in which synthesizing process for the generation ofwide dynamic range, synthesized image is performed within the imagepickup apparatus (camera) by using image data corresponding to twoframes taken at different exposure amounts. The synthesizing process,however, is not necessarily required to be performed within the imagepickup apparatus. It is also possible to provide a personal computer,printer, etc., with a synthesizing function so as to let it performsynthesis of the two frames.

In this embodiment, corresponding to the above manner, the fundamentalconstruction of the image pickup apparatus (camera) is similar to thefirst embodiment shown in FIG. 1. The camera signal processing circuit6, however, is different from that shown in FIG. 2 in that it does notrequire a memory, multiplier, synthesizing circuit, compression circuit,selector, etc., for the performance of synthesizing process and isconstructed as a circuit for performing such ordinary signal processingas r , edge enhancement, etc. Further, DRAM 11 requires a memorycorresponding to two frames and the capacity of the memory card becomesa half the number of images, since data corresponding to two frames mustbe stored for each one image. It should be noted that it is alsopossible to provide detection means for detecting the remaining capacityof the memory card so that, when it is detected that the capacity forstoring data corresponding to two frames is not left in the memory card,data corresponding to one frame can be stored by taking an image of oneframe. It is thereby possible to take image corresponding to an ordinaryone frame even when the remaining capacity is less than that for storingimage data corresponding to two frames.

In the embodiment constructed as the above, two frames of differentexposure amounts are consecutively taken at CCD image pickup device byone release operation to generate image data corresponding to twoframes. Such image data corresponding to two frames are treated as onefile and are temporarily stored to DRAM 11. Further, the image datacorresponding to two frames after signal processing and synthesizinginformation such as exposure amount ratio to be used at the time ofsynthesis are stored as one file to the memory card. The file structureincludes a header 41, short-time exposure image data 42 and long-timeexposure image data 43 as shown in FIG. 15. The header containsinformation for discriminating between short-time exposure image dataand long-time exposure image data, exposure amount ratio, etc. In thesynthesizing process, the file for synthesis is recognized and theheader of the file is referred to by a dedicated software toautomatically generate and display a wide dynamic range, synthesizedimage at personal computer 17 which is connected through the personalcomputer I/F 16 to the image pickup apparatus (camera).

Since two frames at different exposure amounts are thus consecutivelytaken by one release operation at the image pickup apparatus, an imageof moving object such as a person can be taken without being aware oftaking two frames. Further, a complicated processing is not performedwithin the image pickup apparatus due to the fact that the synthesizingprocess of image data of the taken two frames is performed at thepersonal computer externally connected to the image pickup apparatus.Since the processing time is shorter, it becomes possible to shorten theimage taking interval. Furthermore, since the image data correspondingto two frames and the synthesizing information thereof are treated asone file and are subjected to synthesis by a dedicated software at thepersonal computer, a wide dynamic range image can be obtained in asimilar manner as an ordinary taking of image without performing aspecial operation.

The synthesizing processing operation to be performed within thepersonal computer will now be described briefly by way of a flowchartshown in FIG. 16. First, the header of synthesizing process file isreferred to and the exposure amounts of the respective image datacorresponding to two frames at different exposure amounts arediscriminated from each other (step S21). The exposure amount ratio ofthe two image data is computed on the basis of such exposure amounts andunsuitable portions for synthesis as those involving motion of theobject are detected based on the two image data (step S22). From theresult of detection of the above described unsuitable portions forsynthesis, a determination is then made as to whether synthesizingprocess is possible or not (step S23). If a motion has been detected andat the same time if the amount of such motion goes beyond an allowablerange for correction, it is determined as that synthesis is impossibleand the long-time exposure image data is selected (step S24) andoutputted after performing gradation characteristic transform, etc.,(step S25).

On the other hand, if it is determined as that synthesizing is possiblein processing at step S23 for determining whether synthesis is possibleor not, it is then determined with respect to reference pixels whetherthe reference pixels are of portions unsuitable for synthesis or not(step S26). If not of portions unsuitable for synthesis, thesynthesizing process is performed without alteration (step S27). On theother hand, if of portions unsuitable for synthesis, the portionsunsuitable for synthesis are corrected using for example the methodshown in FIG. 4 (step S28). Then, after white balance adjustment (stepS29), the generated, synthesized image is subjected to processing fordynamic range compression and is outputted (step S30).

While the above embodiment has been shown as that for letting anexternally connected personal computer perform synthesizing process, itis also possible that a synthesizing section be similarly provided atthe printer 19 which is connected from the outside through the printerI/F 18 so that the header of the file is referred to at suchsynthesizing section to perform the image synthesizing process and tooutput a synthesized image. In such case, direct printing becomespossible.

As has been described by way of embodiments, in accordance with thefirst aspect of the present invention, since ON/OFF switching control ofthe generation processing operation at the means for generating widedynamic range, synthesized image is performed based on image signalscorresponding to a plurality of previously taken frames, an image pickupapparatus can be achieved as capable of accurately performing ON/OFFswitching control of processing operation for the generation of widedynamic range, synthesized image. In accordance with the second aspectof the invention, since motion detection means is provided so as not toperform the generation processing of synthesized image when a motion ofthe object is detected, it is possible when the object has been moved toprevent a failed, synthesized image from being outputted. In accordancewith the third aspect of the invention, since the motion detection meansand the motion amount determination means and motion correction meansare provided so as to perform synthesizing process with correcting theamount of motion when the detected motion amount is within the allowablerange for correction, it is possible to provide a grater range for theconditions of object of which a wide dynamic range, synthesized imagecan be formed.

Further, in accordance with the fourth aspect of the invention, sinceON/OFF switching of the generation processing operation of wide dynamicrange, synthesized image is performed on the basis of object informationobtained before the taking of image, images can be taken in a mannersuitable for the generation of wide dynamic range, synthesized image. Inaccordance with the fifth aspect of the invention, since images aretaken in a forced strobe emission mode and the generation processing ofsynthesized image is performed when a motion in the object has beendetected at the motion detection means, the motion in the object can ineffect be stopped even of an object involving a motion, making itpossible to readily generate a wide dynamic range, synthesized image. Inaccordance with the sixth aspect of the invention, since the generationprocessing of synthesized image is performed by using a higher shutterspeed when a motion of the object has been detected, it is possible tolessen motion of the object in image signals corresponding to aplurality of frames. Further, since motion of the object can be ineffect reduced in extent even for an object involving motion by using ahigher shutter speed, it is possible to readily generate a wide dynamicrange, synthesized image. In accordance with the seventh aspect of theinvention, since a backlighted condition is detected at an AE detectioncircuit so as to perform the generation processing of wide dynamicrange, synthesized image in a manner suitable at the time of backlightedcondition, a synthesized image can be obtained as suitable for thebacklighted condition.

Further, in accordance with an eighth aspect of the invention, sinceON/OFF switching of the generation processing operation at the means forgenerating wide dynamic range, synthesized image is thus controlledbased on information set in the image pickup apparatus before the takingof image, images can be readily taken in a manner suitable for thegeneration of wide dynamic range, synthesized image without requiringdetection of the conditions for the generation of synthesized image fromtaken images. In accordance with the ninth aspect of the invention,ON/OFF switching of the generation processing operation of synthesizedimage can be performed in a manner linked to the shutter speed and zoominformation so that the generation processing of synthesized image canbe turned OFF to prevent a failed, synthesized image from beingoutputted when the shutter speed is slow or when the zoom ratio is high.In accordance with the tenth aspect of the invention, since means forcorrecting exposure amounts of image signals and ON/OFF switchingcontrol means based on the image taking mode of the exposure amountcorrection means are provided, exposure amount is not always detectedand the exposure amounts can be corrected to obtain a suitablesynthesized image only when there is a possibility of shift from the setexposure amounts of the image signals corresponding to a plurality offrames of different exposure amounts. In accordance with the eleventhaspect of the invention, since the exposure amount correction means iscaused to operate at the time of a strobe image taking mode, a suitablesynthesized image for example without discontinuity in gradation can begenerated even if variance occurs in the amount of strobe emission atthe time of strobe image taking mode.

Further, in accordance with the twelfth aspect of the invention,synthesizing process of the synthesized image is performed at anexternal synthesizing apparatus, it becomes unnecessary to perform acomplicated synthesizing process within the image pickup apparatus sothat time for processing of image taking is shortened and it becomespossible to reduce the image taking interval. In accordance with thethirteenth aspect of the invention, since means for detecting theremaining capacity of recording medium is provided and image signalscorresponding to one frame is recorded when the remaining capacity isless than that for recording image signals corresponding to a pluralityof frames, an ordinary image taking corresponding to one frame can beperformed even when it is impossible to obtain a synthesized image. Inaccordance with the fourteenth aspect of the invention, since imagesignals corresponding to a plurality of frames and the synthesizinginformation thereof are recorded as a single file to the recordingmedium, it is possible to perform synthesizing process by treating themsimilarly as in the case of image signals corresponding to an ordinarysingle frame. In accordance with the fifteenth aspect of the invention,since image signals corresponding to a plurality of frames andsynthesizing information thereof are recorded as a single file in arecording medium from which said file is read out to effect synthesizingprocess, a wide dynamic range synthesized image can be outputtedsimilarly as in the case of an ordinary taken image without a specialoperation. In accordance with the sixteenth aspect of the invention, arecording medium can be provided as that recording a program by whichsynthesizing process can be readily performed of image signalscorresponding to a plurality of frames obtained by taking images of thesame object at a plurality of different exposure amounts.

1. An image synthesizing system including an image pickup apparatus anda synthesizing apparatus, said image pickup apparatus comprising: animage pickup section for taking images of the same object at a pluralityof different exposure amounts to generate image signals corresponding toa plurality of frames of different exposure amounts; and a recordingsection for recording as a single file to a recording medium the imagesignals corresponding to said plurality of frames generated by the imagepickup section and synthesizing information to be used in generating awide dynamic range, synthesized image by synthesizing the image signalscorresponding to the plurality of frames; and said synthesizingapparatus comprising: a synthesizing section for synthesizing the imagesignals corresponding to the plurality of frames contained in said filerecorded in said recording medium on the basis of said synthesizinginformation contained in the file to generate the wide dynamic range,synthesized image.
 2. The image synthesizing system according to claim1, wherein said image pickup apparatus further comprises a detectingsection for detecting the remaining capacity of said recording medium,the remaining capacity of said recording medium being detected by thedetecting section and, if not enough remaining capacity left in therecording medium for recording said image signals corresponding to theplurality of frames of different exposure amounts, image signalscorresponding to one frame being generated by taking image of the objectat a predetermined exposure amount by said image pickup section and theimage signals corresponding to one frame being recorded to saidrecording medium by said recording section.
 3. An image pickup apparatuscomprising: an image pickup section for taking images of the same objectat a plurality of different exposure amounts to generate image signalscorresponding to a plurality of frames at different exposure amounts;and a recording section for recording as a single file to a recordingmedium the image signals corresponding to the plurality of framesgenerated by the image pickup section and synthesizing information to beused in generating wide dynamic range, synthesized image by synthesizingthe image signals corresponding to the plurality of frames.
 4. Asynthesizing apparatus comprising: a reading section for reading a fileout from a recording medium, said file being recorded in said recordingmedium as a single file recording image signals corresponding to aplurality of frames of different exposure amounts obtained by takingimages of the same object at a plurality of different exposure amountsand synthesizing information to be used in generating wide dynamicrange, synthesized image by synthesizing the image signals correspondingto the plurality of frames; and a synthesizing section for generatingwide dynamic range, synthesized image by synthesizing the image signalscorresponding to the plurality of frames contained in the file read outfrom said recording medium by said reading section on the basis of saidsynthesizing information contained in the file.
 5. A recording mediumrecording a program for performing synthesis of image signalscorresponding to a plurality of frames obtained by taking images of thesame object at a plurality of different exposure amounts, said recordingmedium recording a program for reading the image signals correspondingto said plurality of frames contained in one file out from the one filerecording the image signals corresponding to said plurality of frames ofdifferent exposure amounts and synthesizing information to be used ingenerating a wide dynamic range, synthesized image by synthesizing theimage signals corresponding to the plurality of frames, to achieveprocessing for generating wide dynamic range, synthesized image bysynthesizing the readout image signals corresponding to the plurality offrames on the basis of said synthesizing information contained in thesame file.